Weakly-supervised Learning for Fine-grained Emotion Recognition using Physiological Signals

Instead of predicting just one emotion for activity (e.g., video watching), fine-grained recognition enables more temporally precise recognition. Previous works on require segment-by-segment, labels to train the algorithm. However, experiments collect these are costly and time-consuming compared with only collecting label after user watched that stimulus (i.e., post-stimuli labels). To recognize emotions at a finer granularity level when trained labels, we propose an algorithm based Deep Multiple Instance Learning (EDMIL) using physiological signals. EDMIL recognizes valence arousal (V-A) by identifying which instances represent V-A annotated users watching videos. fully-supervised training, weakly-supervised in training stage. The estimated instance gains, indicate probability predict labels. We tested three datasets collected different environments: desktop, mobile HMD-based Virtual Reality, respectively. Recognition results validated self-reports show subject-independent low/neutral/high classification, outperforms state-of-the-art methods. Our find weakly-supervised-learning can reduce overfitting caused temporal mismatch between annotations